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In  zowel  militaire  als  civiele  organisaties  worden  veel  taken  eerder  door  teams  dan  door  individuele 
fiinctionarissen  uitgevoerd.  Ondanks  het  belang  dat  wordt  gehecht  aan  team  optreden  worden 
relatief  weinig  systematische  inspanningen  geleverd  met  betrekking  tot  de  training  van  teams.  Een 
mogelijke  verklaring  hiervoor  is  dat  er  geen  systematiek  is  die  het  ontwikkelen,  uitvoeren  en 
evalueren  van  team  trainingen  ondersteunt.  De  meeste  systematieken  en  richtlijnen  zijn  primair 
gericht  op  het  opleiden  van  de  individuele  functionaris.  Het  gevolg  hiervan  is  dat 
opleidingsontwikkelaars  van  team  trainingen  beperkt  ondersteund  worden  omdat  de  bestaande  (op  de 
individuele  functionaris  gerichte)  systematieken  onvoldoende  de  kenmerken  van  teams  en  team 
optreden  verdisconteren.  Dit  heeft  tot  gevolg  dat  van  bestaande  leermiddelen  die  zijn  ontworpen  met 
als  doel  het  trainen  van  team  vaardigheden,  het  vaak  niet  duidelijk  is  welke  onderwijskundige 
principes  op  welke  manier  toegepast  moeten  worden. 

Team  training  wordt  soms  ten  onrechte  verward  met  team  building  of  cooperatief  leren.  Hoewel  er 
overeenkomsten  zijn,  zijn  er  ook  duidelijke  verschillen.  Een  belangrijk  onderscheid  tussen  team 
training  en  team  building  is,  dat  team  building  meestal  niet  gericht  is  op  de  specifieke  taken  die  een 
team  in  een  bepaalde  context  moet  uitvoeren,  terwijl  team  training  hier  wel  expliciet  aandacht  aan 
besteedt:  er  worden  dus  hele  andere  kennis  en  vaardigheden  behandeld.  In  codperatieve  (onderwijs)- 
leersituaties  is  eerder  sprake  van  een  groep  van  individuen,  dan  van  een  team:  het  leren  is  primair 
gericht  op  het  verwerven  van  vaardigheden  door  het  individu  (bv.  rekenen),  en  minder  nadrukkelijk 
op  het  verwerven  van  ‘sociale’  vaardigheden  om  als  groep  beter  te  functioneren.  Het  leren  gebeurt 
in  de  context  van  een  groep  omdat  verondersteld  wordt  dat  dit  bevorderlijk  is  voor  het  eindresultaat. 
De  onderzoeksvraag  van  deze  studie  is  uit  welke  richtlijnen  een  opleidingsontwikkelsystematiek 
voor  team  trainingen  zou  moeten  bestaan.  Om  deze  vraag  te  kunnen  beantwoorden  wordt  de 
volgende  strategie  voorgesteld:  uitvoeren  van  een  literamursmdie,  uitvoeren  van  een  veldstudie, 
ontwerpen  van  een  prototype,  uitvoeren  van  een  expert-evaluatie,  uitvoeren  van  diverse  experi- 
menten  om  het  prototype  te  testen,  en  toepassen  van  de  systematiek  in  een  of  meerdere 
praktijksituaties.  Deze  strategie  wordt  uitgebreid  toegelicht.  De  resultaten  van  de  literatuurstudie 
(Van  Berio,  1996a)  en  de  veldstudie  (Van  Berio,  1997b)  worden  kort  besproken. 
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Een  voorstel  voor  onderzoek  naar  een  opleidingsontwikkelsystematiek  voor  team  training 
M.P.W.  van  Berio 

SAMENVATTING 

Ondanks  het  belang  dat  wordt  gehecht  aan  team  optreden  worden  relatief  weinig 
systematische  inspanningen  geleverd  met  betrekking  tot  de  training  van  teams.  Een  mogelijke 
verklaring  hiervoor  is  dat  er  geen  systematiek  is  die  het  ontwikkelen,  uitvoeren  en  evalueren 
van  team  trainingen  ondersteunt.  De  onderzoeksvraag  van  deze  studie  is  uit  welke  richtlijnen 
een  opleidingsontwikkelsystematiek  voor  team  trainingen  zou  moeten  bestaan.  Om  deze  vraag 
te  kunnen  beantwoorden  wordt  de  volgende  strategie  voorgesteld:  uitvoeren  van  een 
literatuurstudie,  uitvoeren  van  een  veldstudie,  ontwerpen  van  een  prototype,  uitvoeren  van 
een  expert-evaluatie,  uitvoeren  van  diverse  experimenten  om  het  prototype  te  testen,  en  het 
toepassen  van  de  systematiek  in  een  of  meerdere  praktijksituaties.  Deze  strategie  wordt 
uitgebreid  toegelicht.  De  resultaten  van  de  literatuurstudie  (Van  Berio,  1996a)  en  de  veld¬ 
studie  (Van  Berio,  1997b)  worden  kort  besproken. 
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1  BACKGROUND 

Technological  developments  have  resulted  in  more  sophisticated  and  complex  systems  in 
which  humans  have  to  operate.  These  systems  are  characterized  by  a  highly  dynamic  and 
sometimes  hostile  environment,  the  variation  of  (often  conflicting)  goals,  the  incompleteness, 
uncertainty  and  ambiguity  of  information,  and  the  involvement  of  teams  of  officers  with 
members  having  different  roles  and  responsibilities  (Rouse,  Cannon-Bowers  &  Salas,  1992). 
Many  tasks  can  not  be  performed  by  just  one  operator,  but  need  to  be  accomplished  by  a  team 
of  several  operators.  A  team  is  defined  as  a  group  of  two  or  more  people,  with  a  common 
goal,  a  specific  role  assignment,  and  tasks/activities  that  are  interdependent  (Dyer,  1984).  A 
team  makes  decisions  in  the  context  of  a  larger  task,  team  members  have  specialized 
knowledge  and  skills  relevant  to  the  task  and  decisions,  and  the  task  conditions  under  which 
the  team  operates  often  include  high  workload  and  time-pressure  (Orasanu  &  Salas,  1993; 
Salas,  Bowers  &  Cannon-Bowers,  1995). 

In  both  military  and  civil  organizations  many  tasks  are  performed  by  teams  rather  than 
individual  officers.  A  fire-brigade,  a  surgical  team,  a  tank  platoon,  and  a  Stinger  group  are 
examples  of  such  teams.  Despite  the  acknowledgment  of  the  importance  of  team  performance 
and  team  training,  relatively  few  endeavours  have  been  undertaken  to  train  teams  in  a 
systematic  way.  Team  training  is  rarely  being  conducted  as  a  separate  training  (except  bridge 
and  crew  resource  management)  with  the  argument  that  functioning  as  a  team  can  best  be 
learned  in  the  operational  environment,  and  after  each  member  is  being  trained  for  the 
individual  tasks.  Accident  reports  show  that  this  ‘on-the-job  team  training’  does  not  seem  to 
be  the  most  effective  and  efficient  way  of  preparing  the  team  for  its  mission  and  tasks  (Salas, 
Cannon-Bowers  &  Johnston,  1995).  This  process  could  be  enhanced  by  carefully  developed 
team  training  systems. 

A  major  obstacle  in  the  process  of  developing  team  training  systems  is  that  most 
methodologies  and  guidelines  for  developing  training  systems  are  aimed  at  the  individual 
trainee  (Armstrong  &  Reigeluth,  1991).  A  coherent  methodology  for  developing  team  training 
systems  is  still  lacking  (Miller,  Guerette  &  Morgan,  1987;  Guerette  et  al.,  1987). 
Consequently,  developers  of  team  training  systems  must  often  resort  to  the  use  of 
recommendations  and  guidelines  geared  to  the  development  of  individual-centered  training 
systems.  However,  these  may  be  insufficient  given  the  different  nature  and  characteristics  of 
teams  and  team  performance.  Related  to  this  point  is  the  fact  that  most  instructional  devices 
are  developed  for  training  individual  officer’s  skills.  Regarding  complex  learning 
environments  developed  for  team  training,  e.g.  SIMNET  (Alluisi,  1991)  and  CATT 
(Combined  Arms  Tactical  Trainer),  it  is  often  not  clear  which,  and  how,  instructional 
principles  should  be  applied  to  train  the  team  (Salas  &  Cannon-Bowers,  in  press). 

In  many  cases,  the  to  be  trained  team  is  presented  with  instruction  and  feedback  at  an  ad-hoc 
basis,  leaving  the  content  and  timing  of  instruction  and  feedback  to  the  initiative  of  the 
instructor  (Guerette  et  al.,  1987).  Consequently,  instructors  devote  much  time  and  effort  in 


6 


on-line  determining  the  most  effective  way  of  presenting  information  and  measuring  the  team 
performance,  and  the  most  adequate  way  of  providing  feedback.  This  process  could  be  more 
effective  and  efficient  if  a  more  formal  and  systematic  methodology  for  developing  and 
monitoring  team  training  would  be  available  (Miller,  Guerette  &  Morgan,  1987). 

More  than  fifteen  years  ago  several  authors  have  indicated  the  need  for  a  systematic 
development  of  team  training  (Kribs,  Thurmond  &  Marks,  1977;  Rizzo,  1980;  Thurmond, 
1980).  The  development  of  team  training  systems  is  a  complex  and  costly  enterprise.  A 
methodology  for  systematically  developing  team  training  could  help  to  reduce  these  costs.  It 
can  eliminate  unnecessary  or  ineffective,  but  costly,  training  practices  that  do  not  contribute 
to  the  learning  of  team  skills.  Furthermore,  the  costs  associated  with  poor  team  performance 
in  operational  environments  due  to  inadequate  training,  are  probably  very  high. 

Team  training  is  often  confused  with  the  concepts  ‘team  building’  and  ‘cooperative  learning’: 
in  chapter  2  the  field  of  team  training  research  will  be  defined.  In  chapter  3  the  research 
question  of  this  study  is  delineated.  The  proposed  strategy  to  follow  in  order  to  answer  the 
research  question,  is  presented  in  chapter  4.  This  is  followed  by  a  discussion  of  the  literature 
study  (chapter  5)  and  the  field  study  (chapter  6). 


2  DEFINING  THE  FIELD  OF  RESEARCH 

Many  factors  have  an  influence  on  the  effectiveness  of  teams,  like,  amongst  other  things,  the 
number  of  team  members,  the  allocation  of  tasks  within  a  team,  the  way  personnel  is 
rewarded,  the  role  of  the  team  commander,  the  way  the  team  is  embedded  in  the  organization, 
the  selection  of  adequate  team  members,  team  building  activities,  and  team  training.  This 
research  focuses  on  team  training.  The  concept  ‘team  training’  is,  however,  often  confused 
with  the  concept  ‘team  building’.  Although  the  ultimate  goal  of  both  team  training  and  team 
building  is  the  same  (i.e.  improving  team  performance),  there  are  some  considerable 
differences  between  the  two.  Also,  ‘team  training’  is  sometimes  wrongly  conceived  of  as 
‘cooperative  learning’.  Again,  there  are  similarities,  but  distinctions  as  well.  In  this  chapter 
the  differences  and  relationships  between  team  training  on  the  one  hand,  and  team  building 
and  cooperative  learning  on  the  other  hand,  are  discussed,  with  the  purpose  to  define  the  field 
of  research  described  here  (Van  Berio,  1997a). 

Training  the  members  of  a  team  can  be  primarily  aimed  at  skills  required  for  adequate  team 
task  performance,  or  at  social  skills  required  for  functioning  as  a  group  (taskwork  vs 
teamwork:  Morgan  et  al.,  1986;  Glickman  et  al.,  1987).  ‘Team  training’  is  focusing  on  the 
taskwork,  while  ‘team  building’  has  its  focus  on  teamwork.  Team  building  is  not  specifically 
aimed  at  tasks  a  team  has  to  perform  in  the  operational  task  environment,  while,  on  the  other 
hand,  team  training  explicitly  focuses  on  these  tasks. 
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Both  team  training  and  team  building  are  kinds  of  training  a  team,  but  each  type  is  aiming  at 
different  knowledge  and  skills  required  for  adequate  team  performance. 

‘Training’  is  a  systematic  enterprise  to  enable  the  trainees  to  acquire  the  knowledge,  skills  and 
attitudes  required  for  adequate  task  performance.  Based  on  instructional  objectives  the 
contents  and  structure  of  the  training  is  determined  in  such  a  way  that  these  objectives  will  be 
met.  ‘Team  building’,  however,  is  more  like  a  set  of  activities  with  the  intention  to  give 
(groups  of)  individuals  a  deeper  understanding  of  their  behaviour  and  their  interpersonal 
relationships.  Although  a  general  framework  can  be  formulated  in  advance,  in  most  cases  the 
specific  content  of  the  team  building  intervention  is  more  dependent  on  discussions,  between 
trainees  and  with  the  trainer,  during  the  intervention. 

A  final  distinction  between  team  building  and  team  training  is  that  team  building  activities  are 
always  performed  with  the  persons  comprising  the  actual  team:  in  view  of  the  general  goal  of 
team  building  it  would  be  undesirable  to  involve  other  persons  into  this  effort.  Team  training 
does  not  necessarily  need  to  be  conducted  with  the  actual  team  members:  every  team  member 
gets  a  training  aimed  at  the  own  role  and  the  own  tasks  within  the  team  (independent  of  who 
will  be  the  other  team  members),  so  it  is  not  a  disadvantage  per  se  that  other  persons  play  the 
roles  of  other  team  members. 

A  cooperative  learning  situation  resembles  daily  practice  in  which  persons  work  together  with 
other  persons.  But  learning  in  groups  is  not  the  same  as  team  training.  In  cooperative  learning 
situations  it  are  individuals  that  learn  together,  rather  than  a  team  (see  chapter  1).  In 
accordance  with  the  cooperative  learning  perspective,  learning  in  groups  benefits  the 
acquisition  of  knowledge  and  skills  by  the  individual  trainee  (e.g.  arithmetic),  and,  as  a  side 
effect,  the  acquisition  of  ‘social’  skills  as  well  (e.g.  collaborating  with  each  other).  In  fact, 
trainees  are  examined  individually  in  order  to  assess  whether  they  have  acquired  these 
individual  knowledge  and  skills.  The  focus  is  on  individuals,  and  the  learning  takes  place  in  a 
group  for  didactical  purposes.  The  acquisition  of  social,  interpersonal  skills  rather  seems  a 
secondary  effect  of  this  kind  of  instruction,  and  has  not  been  systematically  demonstrated 
(Erkens,  1997). 


3  RESEARCH  QUESTION 

The  objective  of  any  training  system  is  to  change  the  competencies  of  the  trainees  so  they  can 
perform  their  tasks  in  the  operational  environment  effectively  and  efficiently.  The  task,  the 
context  in  which  it  has  to  be  accomplished,  and  the  required  skills  make  up  the  conditions  for 
the  instructional  systems  development  process.  A  description  of  the  task  that  has  to  be 
learned,  and  the  way  in  which  this  will  be  implemented  in  a  training  program,  is  the  output  of 
several  consecutive  steps.  These  steps  can  be  subsumed  under  the  headings  of  analysis, 
design,  implementation,  and  evaluation.  Exactly  how  to  follow  the  distinct  steps  that  comprise 
these  phases,  is  depicted  in  guidelines.  Developing  team  training  systems  could  include  the 
same  phases  of  development:  tasks  as  well  as  the  requisite  knowledge,  skills  and  attitudes 
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have  to  be  analysed;  instruction,  practice  and  training  devices  have  to  be  designed;  the 
training  system  has  to  be  field  tested;  the  training  system  must  provide  for  assessment  of  the 
trainees,  and  the  training  system  must  be  evaluated  itself.  However,  the  guidelines  to  follow 
the  steps  could  be  somewhat  different.  For  instance,  analysing  a  team  task  probably  poses 
different  demands  than  analysing  an  individual  task.  Also,  designing  scenarios  for  instruction 
and  practice  of  the  team  is  somewhat  more  complicated  than  scenario  design  for  individual 
trainees.  Team  training  devices  have  other  requirements  (e.g.  communication  processes)  than 
training  devices  for  individual  trainees.  Further,  assessing  the  team’s  performance  is  more 
complex. 

A  team  training  methodology  should  be  a  coherent  set  of  guidelines,  based  on  principles  of 
learning  and  training,  for  developing,  designing  and  delivering  instruction  to  enhance  and 
maintain  team  performance  in  the  operational  task  environment.  It  should  involve  creating  a 
learning  environment  in  which  the  team  members  can  acquire  and  practice  the  necessary 
knowledge,  skills  and  attitudes  required  for  adequate  team  performance.  Adequate 
diagnosing,  assessing  and  remediating  the  team’s  performance  should  be  essential  features  of 
a  training  system  to  be  effective  (Salas  &  Cannon-Bowers,  in  press). 

Given  the  need  for,  and  the  demands  made  upon  a  methodology  for  developing  team  training 
systems,  the  basic  research  question  to  be  answered  in  this  project  is: 

Which  guidelines  should  be  included  in  a  methodology  supporting  the  systematic 
development  of  team  training  systems? 

In  answering  this  question  insight  needs  to  be  gained  into  several  topics: 

•  what  is  described  in  the  literature  about  the  design  and  development  of  team  training? 

•  how  is  team  training  being  designed  and  developed  within  the  military? 

•  what  requirements  should  advanced  training  devices  meet  in  optimizing  team  training? 

•  how  should  training  of  individual  operators  and  team  training  be  harmonized? 

•  how  does  the  development  of  shared  mental  models  affect  team  training  (Schaafstal  & 
Bots,  1997)? 


4  STRATEGY 

In  this  chapter  the  proposed  strategy  to  follow,  in  order  to  answer  the  research  question,  is 
presented.  The  proposed  strategy  consists  of  11  consecutive  steps,  each  representing  a  distinct 
phase  in  the  research  project. 

Step  1:  Conduct  a  literature  study.  See  chapter  5  for  the  results  of  the  literature  study. 

Step  2:  Conduct  a  field  study. 

a  Conduct  interviews  with  staff  members  of  military  training  centres  where  the 
training  developers  and  trainers  themselves  are  being  trained. 
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b  Conduct  interviews  with  the  training  developers  and  trainers  of  the  military 
training  centres. 

c  Conduct  interviews  with  training  practitioners  in  civil  organizations, 
d  Attend/observe  team  training  programs.  If  there  would  be  few  possibilities  to 
do  this  in  the  Netherlands,  the  field  study  will  be  extended  to  the  USA  (e.g. 
National  Training  Centre,  Fort  Knox,  Naval  Air  Warfare  Center), 
e  Review  the  reports  of  visits  other  colleagues  have  made  to  several  military 
training  centres. 

The  field  study  is  described  in  more  detail  in  chapter  6. 

Step  3:  Integrate  the  results  of  both  literature  and  field  study,  and  develop  a  (paper- 

based)  prototype  of  a  methodology  for  developing  team  training  systems.  This 
prototype  methodology  has  to  contain  guidelines  as  concrete  as  possible. 

Step  4:  Conduct  a  first  expert-evaluation  of  the  prototype.  Experts  could  be  from  the 

military,  from  TNO-HFRI,  or  from  external  organizations  (civil,  universities). 
Possible  methods  for  this  evaluations  can  be:  interview,  evaluation  checklist, 
group  discussion,  Delphi,  or  combinations  of  these  methods.  No  decision  has 
been  made  regarding  the  implementation  of  this  step  yet. 

Step  5:  Test  the  prototype  of  the  methodology  in  various  experiments. 

Experiment  1 : 

Goal:  The  purpose  of  the  first  experiment  is  to  use  the  subjects  as  co-constructors 
of  the  methodology:  because  they  are  experts,  they  will  probably  have  valuable 
comments  on  the  prototype.  The  following  approach  is  suggested: 

Subjects:  The  subjects  are  working  in  civil  organizations  and  must  have  at  least 
four  to  five  years  of  experience  as  instructional  designers.  The  advantage  of 
using  experts  is  that  they  can  give  useful  comments  concerning  the  usability  of 
the  methodology.  The  risk  of  using  military  subjects  is  the  variability  in  level 
of  experience  and  expertise:  usually  an  officer  holds  a  position  for  only  three 
years.  Using  the  comments  and  remarks  of  the  expert-subjects  will  benefit  the 
construction  of  a  more  advanced  prototype  of  the  methodology. 

Task  and  materials:  The  subjects  are  asked  to  (individually)  develop  a  blueprint 
of  a  team  training  program  based  on  the  guidelines  of  the  prototype.  The  team 
task  being  used  is  TANDEM^:  TActical  Navy  DEcision  Making  task, 
developed  by  and  obtained  from  the  Naval  Training  Systems  Center  (Florida). 
The  advantage  of  this  laboratory  team  task  is  that  subject-matter  knowledge  is 
being  controlled  for. 


^  Provided  that  TANDEM  is  a  really  complex  team  task,  and  the  team  members  should  acquire  considerable 
specific  knowledge  and  skills  for  an  adequate  task  performance.  If  not,  another  experimental  team  task  has  to  be 
developed  (e.g.  a  modification  of  Space  Fortress). 
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Procedure:  The  subjects  receive  documentation  explaining  the  TANDEM-task, 
and  a  (brief)  demonstration  of  this  team  task.  Next,  the  subjects  have  about  V-h 
hours  to  design  a  blueprint  of  a  TANDEM-training  program.  They  have  to 
think  aloud,  and  are  allowed  to  make  notes.  In  this  way  a  baseline 
measurement  can  be  performed.  After  the  subjects  have  designed  a  training 
program,  the  prototype  of  the  methodology  is  discussed.  This  discussion  is 
initiated  by  using  a  walk-through  procedure,  in  which  the  researcher  guides  the 
subjects  through  the  (paper-based)  methodology.  During  this  phase,  the 
researcher  asks  questions  regarding  the  meaning  and  interpretation  of  several 
concepts,  steps  and  guidelines.  Based  on  the  subjects’  experiences  of  the 
experimental  task,  they  are  expected  to  give  feedback  and  comments  on  the 
prototype  of  the  methodology. 

Experiment  2: 

Goal:  The  purpose  of  the  second  experiment  is  to  determine  to  what  extent 
subjects  have  a  corresponding  interpretation  of  the  prototype:  do  subjects 
interpret  the  distinct  steps  in  a  similar  way  (reliability)?  The  following 
approach  is  suggested: 

Subjects:  In  this  experiment  the  subjects  are  military  instructional  designers  with 
about  two  to  three  years  of  experience.  In  this  step  the  target  group  (the 
military)  will  use  the  methodology. 

Task  and  materials:  The  subjects  are  asked  to  (individually)  develop  a  blueprint 
of  a  team  training  program  based  on  the  guidelines  of  the  prototype.  The  team 
task  being  used  is  TANDEM:  TActical  Navy  DEcision  Making  task,  developed 
by  and  obtained  from  the  Naval  Training  Systems  Center  (Florida).  The 
advantage  of  this  laboratory  team  task  is  that  subjects’  subject-matter 
knowledge  is  being  controlled  for. 

Procedure:  The  subjects  receive  instruction  regarding  the  prototype- 
methodology.  This  instruction  is  provided  using  a  walk-through  procedure,  in 
which  the  researcher  guides  the  subject  through  the  (paper-based) 
methodology.  During  this  phase,  the  researcher  asks  questions  regarding  the 
meaning  and  interpretation  of  several  concepts,  steps  and  guidelines.  Subjects 
receive  documentation  explaining  the  TANDEM-task,  and  a  (brief) 
demonstration  of  this  team  task.  The  subjects  have  about  IVi  hours  to  design  a 
blueprint  of  a  TANDEM-training  program.  They  have  to  think  aloud,  and  are 
allowed  to  make  notes.  The  think-aloud  protocols  will  be  analysed  (Perez, 
Fleming  Johnson  &  Emery,  1995).  After  the  session  the  subjects  are  asked  to 
complete  an  evaluation  form. 

Experiment  3: 

Goal:  The  purpose  of  the  third  experiment  is  to  determine  the  effect  of  the 
methodology  in  an  experimental  setting.  The  following  approach  is  suggested: 
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Subjects:  In  this  experiment  the  subjects  are  ‘Educational  science’  students.  They 
do  not  have  broad  experience  and  expertise  in  developing  instruction,  but 
understand  the  relevant  basic  knowledge  and  skills. 

Task  and  materials:  The  subjects  are  asked  to  (individually)  develop  blueprints  of 
two  team  training  programs.  The  first  team  task  being  used  is  TANDEM,  the 
second  team  task  is  a  modified  version  of  Space  Fortress. 

Procedure:  The  subject  are  divided  into  two  groups.  The  first  group  is  asked  to 
develop  a  blueprint  of  a  TANDEM  training  program,  without  the  support  of 
the  methodology.  After  the  development  of  the  blueprint,  the  subjects  receive 
instruction  in  the  methodology.  After  this  instruction  the  group  must  develop  a 
Space  Fortress  training  program.  The  second  group  follows  the  same 
procedure,  but  first  develops  a  blueprint  of  a  Space  Fortress  training  program, 
followed  by  a  blueprint  of  a  TANDEM  training  program.  In  this  way  for  both 
team  tasks  the  effect  of  the  methodology  on  the  quality  of  the  blueprints  can  be 
measured. 

Step  6:  Modify,  if  necessary,  the  prototype-methodology  based  on  the  results  of  the 

experiments. 

Step?:  Select  a  real-life  case^.  The  goal  of  the  case  study  is  to  test  the  prototype- 

methodology  in  a  field  setting.  The  case  should  meet  the  following  requirements: 
a  the  training  centre  is  willing  to  alter,  if  necessary,  their  way  of  developing  and 
executing  (team)  training  programs 

b  the  team  training  program  is  performed  on  a  regular  basis.  This  will  provide  a 
reference  point. 

c  it  should  be  possible,  from  a  research  perspective,  to  gain  insight  in  process 
and  product  variables  relating  to  instructional  system  development  (see  step  9). 
This  insight  will  be  obtained  from  studying  relevant  documents  and  by 
interviewing  the  instructional  designer(s)  and  trainer(s).  The  information  has  to 
be  assessed  using  predefined  criteria  that  must  be  identical  to  the  criteria  used 
in  the  evaluation  of  the  ‘new’  team  training  program;  if  this  information  is  hard 
to  obtain,  a  review  of  the  current  ISD-process  conform  step  9  should  be 
considered.  In  this  way  both  a  pre-  and  a  post  measurement  can  be  realized 
(One-Group  Pretest  -  Posttest  Design,  see:  Cook  &  Campbell,  1979). 

Step  8:  Give  instruction/training  to  the  developers  to  make  sure  the  methodology  will  be 

used  in  a  proper  way.  Construct  a  manual  and/or  provide  computer-based 
support. 


^  If  steps  7  through  9  are  too  difficult  to  arrange  practically  or  too  time-consuming,  the  use  of  an  experimental 
environment  should  be  considered. 
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Step  9:  Determine  the  quality  of  the  new  methodology  based  on  the  experiences  obtained 

during  the  case  study.  The  quality  can  be  determined  on  two  dimensions:  process 
and  product. 

/  -  Process: 

Variables 

a  the  way  intermediate  products  have  been  produced 
b  the  total  time  devoted  to  the  development  process 
c  aspects  on  which  the  methodology  is  not  sufficient 
d  aspects  on  which  the  methodology  is  sufficient 

Methods 

a  frequently  interviewing  the  instructional  developers  (and  trainers) 
b  attend  the  meetings  between  developers  (and  trainers) 

c  let  the  developers  keep  a  ‘log-book’  in  which  they  write  down  their 
experiences.  This  logbook  could  be  integrated  in  a  computer-based  software 
tool  supporting  the  methodology. 

11 -PRODUCT: 

Variables 

a  the  results  of  the  analyses 
b  the  quality  of  the  intermediate  products 
c  the  quality  of  the  final  result  (the  training  program) 
d  suggested  changes/modifications  of  the  methodology 
e  the  students’  results 
f  the  total  training  time 

Methods 

a  let  trainers  complete  a  questionnaire  (possibly  with  additional  interviews) 
b  let  students  complete  a  questionnaire  (possibly  with  additional  interviews) 
c  assess  the  intermediate  and  final  documents  (results  of  analyses,  instructional 
objectives,  blueprint  of  the  program,  training  scenarios)  by  independent 
(external)  reviewers.  A  checklist  has  to  be  developed,  as  well  as  the  criteria, 
d  implement  the  team  training  and  measure  the  performance  of  the  students/team. 

The  results  of  the  new  approach  can  be  compared  with  the  results  of  the 
traditional  approach.  In  this  way  a  measurement  of  the  effect  of  the  new 
methodology  can  be  established. 

Step  10:  Modify,  if  necessary,  the  prototype-methodology  based  on  the  results  of  the  case 
study. 
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Step  11:  Select  a  new  case,  and  follow  steps  8  through  10  again. 

The  aim  is  to  develop  a  methodology  as  general  (and  useful)  as  possible,  suited 
for  a  wide  range  of  teams  and  team  tasks.  This  could  be  achieved  by  conducting 
several  case  studies.  Striving  for  generalizability  is  also  achieved  by  conducting  a 
rather  broad  literature  and  field  study. 


5  LITERATURE  STUDY 

A  literature  study  has  been  conducted  in  order  to  analyse  to  what  extent  guidelines  are 
available  that  specifically  address  the  development  of  team  training  systems.  In  this  chapter, 
the  literature  study  is  briefly  discussed:  for  a  more  detailed  description  and  evaluation  of  the 
guidelines,  and  discussion  of  the  results,  see  Van  Berio  (1996a). 

Developing  instructional  systems  roughly  consists  of  four  distinct  phases:  analysis,  design, 
implementation,  and  evaluation.  With  regard  to  team  training,  the  analysis  phase  is  partly 
described  by  the  MAP  system  (Levine  et  al.,  1988),  and  the  TEAM  model  (Morgan  et  ah, 
1986;  Salas,  Morgan  &  Glickman,  1987).  The  Multiphase  Analysis  of  Performance  (MAP) 
system  consists  of  a  taxonomy  for  team  training.  First,  the  instruction  can  be  aimed  at  the 
individual  team  member  or  at  the  team  as  a  whole.  Next,  both  individual  and  team  can  be 
either  experienced  or  inexperienced.  Finally,  training  can  be  directed  towards  interpersonal 
skills  or  production  skills.  Combining  these  levels  results  in  an  eight-cell  job  analysis 
taxonomy  for  team  training  (Levine  et  ah,  1988;  Levine  &  Baker,  1991).  For  each  cell,  the 
authors  have  indicated  the  descriptors  which  characterize  the  team  task,  the  sources  of  data 
that  can  be  regarded,  the  most  adequate  methods  of  data  collection,  and  the  methods  of  data 
analysis. 

The  Team  Evolution  and  Maturation  (TEAM)  model  is  based  on  the  assumption  that  a  team 
passes  through  several  phases  in  becoming  a  proficient  team  (Morgan  et  ah,  1986;  Salas, 
Morgan  &  Glickman,  1987).  As  the  training  progresses  the  team  learns  more  about  the 
operational  team  task  itself,  the  demands  posed  by  the  task  environment,  the  abilities  and 
characteristics  of  the  other  team  members,  and  about  working,  coordinating,  and 
communicating  collectively.  This  process,  the  maturation  from  an  unskilled  and  immarnre 
team  towards  a  skilful  and  mature  team,  has  been  classified  into  nine  phases.  In  the  TEAM 
model  a  specific  distinction  is  made  between  taskwork  and  teamwork.  To  attain  an  optimum 
effect  of  the  team  training  it  is  assumed  that  both  taskwork  and  teamwork  should  first  be 
trained  separately.  Gradually  these  skills  should  be  trained  more  integrated,  so  eventually  both 
skills  can  be  applied  in  performing  the  team  task. 

The  phases  of  both  analysis  and  design  are  described  by  the  TIP  theory  (Armstrong  & 
Reigeluth,  1991)  and  by  the  propositions  for  team  training  by  Cannon-Bowers  et  al.  (1995b). 
The  Team  Instructional  Prescriptions  (TIP)  theory  (Armstrong  &  Reigeluth,  1991)  contains 
four  elements:  the  desired  output,  the  conditions,  the  instructional  methods,  and  a  set  of 
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guidelines  in  which  output,  conditions  and  methods  are  being  integrated.  The  result  of  team 
training  is  twofold:  it  is  directed  at  both  teamwork  and  the  team  task  (Salas,  Morgan  & 
Glickman,  1987).  The  conditions  influencing  team  training  are  comprised  of  three  types  of 
variables;  the  team  development  phases  (Morgan  et  al.,  1986),  the  task  dimensions 
(interdependent  vs  independent  tasks,  and  procedural  vs  transfer  tasks)  and  the  relationships 

between  tasks  (superordinate,  coordinate,  subordinate).  The  instructional _ methods  are 

discussed  at  three  levels  of  strategy:  macro,  mid-level,  and  micro  strategies.  Result, 
conditions  and  instructional  methods  are  integrated  into  a  coherent  set  of  prescriptive 
guidelines.  The  TIP  theory  contains  three  models,  each  representing  a  team  development 
phase.  Each  model  consists  of  a  combination  of  task  relationships  and  the  task  dimensions. 
Each  model  embraces  the  instruction  and  practice  regarding  the  role  the  individual  team 
member  has  within  the  team:  the  more  the  team  matures,  this  role-instruction  and  practice  will 
be  more  specific. 

Cannon-Bowers  et  al.  (1995b)  presented  a  framework  providing  four  types  of  team 
competencies  based  on  simational  and  task  characteristics  (context-driven,  team-contingent, 
task-contingent,  transportable).  Based  upon  this  framework  the  authors  offer  sixteen 
propositions  regarding  the  nature  of  team  training  required  for  developing  specific 
competencies  in  teams  and  regarding  the  most  successful  strategies.  The  propositions  fall  into 
two  related  categories:  those  that  involve  the  manner  in  which  the  task  and  situational  (i.e. 
environmental)  characteristics  influence  the  nature  of  the  team’s  competency  requirements, 
and  those  that  link  the  categorization  of  team  competencies  to  training  requirements  and 
strategies. 

Guidelines  for  developing  training  scenarios  during  the  design  phase  are  presented  by  Prince 
et  al.  (1993)  and  Schank  et  al.  (1993/1994).  Based  on  existing  guidelines  (Federal  Aviation 
Administration,  1990;  Lauber  &  Foushee,  1981)  and  practical  training  experience.  Prince  et 
al.  (1993)  presented  guidelines  for  simulator  scenario  development  with  respect  to  crew 
resource  management  behaviour  training.  The  guidelines  are  separated  into  five  categories: 
scenario  overview,  objectives,  realism,  role  of  the  facilitator,  and  technical  tips. 

Schank  and  his  co-workers  (1993/1994)  focus  on  so-called  Goal-Based  Scenarios  that  can  be 
performed  by  the  trainees  in  a  computer-based  learning-by-doing  environment.  A  Goal-Based 
Scenario  (GBS)  is  defined  as  such  to  emphasize  that  a  training  scenario  should  be  task- 
oriented,  with  clearly  specified  objectives.  Schank  et  al.  (1993/1994,  p.  322/3)  identify  seven 
general  criteria  that  a  GBS  design  should  meet:  thematic  coherence,  realism/richness, 
control/empowerment,  challenge  consistency,  responsiveness,  pedagogical  goal  support,  and 
pedagogical  goal  resources.  A  GBS  consists  of  several  components  that  should  be  identified 
successively:  mission,  mission  focus,  cover  story,  and  scenario. 

The  phases  of  design  and  implementation  are  partially  described  by  the  TIPM  model  (Guerette 
et  al.,  1987;  Miller,  Guerette  &  Morgan,  1987),  and  the  Teamwork  and  Instructional 
Characteristics  Checklists  (Swezey,  Llaneras  &  Salas,  1992).  The  checklists  also  partly 
describe  the  evaluation  phase.  Based  on  the  experiences  with  the  previously  described  TEAM 
model,  the  Team  Instructional  Processes  Model  (TIPM)  has  been  developed  (Guerette  et  al.. 
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1987;  Miller,  Guerette  &  Morgan,  1987).  The  team  development  phases  as  described  by 
Morgan  et  al.  (1987)  are  linked  with  specific  training  strategies.  The  TIPM  contains  ten  steps: 
(1)  pretraining  capability  assessment,  (2)  preliminary  assessment  of  needs,  (3)  determination 
of  training  approach,  (4)  information  presentation,  (5)  mission  performance,  (6)  evaluation 
and  debrief,  (7,  8  and  9)  more  specific  training  (cm  steps  3,  4  and  5),  and  (10)  evaluation  and 
joint  debrief. 

Swezey,  Llaneras  and  Salas  (1992)  constructed  two  checklists  supporting  the  organization  and 
presentation  of  guidelines  regarding  the  development  and  evaluation  of  team  training  systems: 
the  Teamwork  Characteristics  checklist  (30  items)  and  the  Instructional  Characteristics 
checklist  (41  items).  The  first  step  in  using  each  checklist  is,  for  each  item  on  the  list,  to 
indicate  whether  the  training  program  includes  the  respective  (teamwork  or  instructional) 
characteristic.  This  step  is  followed  by  scoring  the  importance  of  each  characteristic  on  a 
5-point  rating  scale  (0  =  not  important;  4  =  extremely  important),  regardless  of  the  result  of 
the  first  step.  By  combining  these  two  results,  shortfalls  in  the  training  system  can  be  easily 
identified. 

Given  the  definition  of  a  team  training  methodology,  the  four  general  phases  of  instructional 
systems  development,  and  the  state-of-the-art  as  described  in  the  literature,  an  inventory  can 
be  made  up  of  the  missing  knowledge  in  producing  an  integrated  methodology  for  developing 
team  training  systems. 

First  of  all,  it  is  important  that  the  performance  deficiencies  of  a  team  are  identified.  Such  a 
tool  for  conducting  a  performance  analysis  is  still  lacking  (Salas  &  Cannon-Bowers,  in  press). 
Based  upon  these  performance  deficiencies,  the  training  needs  of  the  team  must  be  analysed 
(Salas,  Bowers  &  Cannon-Bowers,  1995)  and  a  selection  of  the  to-be-trained  tasks  has  to  be 
made  (Bowers,  Baker  &  Salas,  1994).  Finally,  a  tool  has  to  be  developed  by  which  the  task 
demands  and  the  training  needs  could  be  linked  with  specific  training  strategies,  including 
proper  performance  assessment  methods  and  strategies  for  remediation  (Salas,  Bowers  & 
Cannon-Bowers,  1995). 

There  still  is  not  a  connection  node  between  the  way  a  team  matures,  and  how  the  training 
system  should  correspond  with  this  process  (Salas  &  Cannon-Bowers,  in  press).  Procedures 
and  tools  for  optimizing  training  strategies  based  on  key  team  variables  should  be  developed 
(Guerette  et  al.,  1987).  Not  just  the  process  remains  unclear;  the  way  in  which  the  training 
system  should  take  into  account  the  input  variables  (e.g.  characteristics  of  the  trainees  and  the 
environment)  remains  unclear  as  well.  A  pre-training  diagnostic  instrument  to  assess  the 
team’s  abilities  needs  to  be  developed  (Guerette  et  al.,  1987).  More  specifically,  gaining 
insight  into  the  relationship  between  team  attitudes  (e.g.  team  cohesiveness)  and  team 
performance  can  benefit  the  training  of  teams  (Salas,  Bowers  &  Cannon-Bowers,  1995).  The 
same  holds  true  for  the  impact  of  the  organizational  context  on  the  training  system.  The 
required  output  of  a  team  training  system  (e.g.  what  constitutes  a  good  team  performance) 
needs  to  be  defined  in  a  more  complete  way  (Cannon-Bowers  et  al.,  1995a;  Armstrong  & 
Reigeluth,  1991;  Salas,  Morgan  &  Glickman,  1987;  Miller  et  al.,  1987;  Guerette  et  al., 
1987). 
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A  methodology  for  developing  team  training  systems  should  be  based  on  principles  of 
learning  and  training.  Ascertaining  which  principles  are  prevailing,  and  how  these  principles 
should  be  applied  in  team  training  systems,  is  a  major  research  question  (Salas  &  Cannon- 
Bowers,  in  press). 

Complex,  technologically  advanced  learning  environments  are  being  developed  for  team 
training.  As  already  indicated,  it  is  often  not  clear  which,  and  how,  instructional  principles 
should  be  applied  to  team  training  systems.  The  application  of  new,  advanced  technologies 
like  multi  media,  intelligent  tutoring  systems,  teleconferencing,  and  distributed  interactive 
simulation  should  be  smdied  in  a  more  profound  way. 

Guidelines  for  designing  training  scenarios  are  almost  non-existent.  Two  sets  of  guidelines 
have  been  identified  (Prince  et  al.,  1993;  Schank  et  al.,  1993/1994)  both  comprising  valuable 
components.  An  integrated  set  of  guidelines  for  designing  scenarios,  encompassing  the  entire 
process  of  scenario  development,  is  still  lacking. 

Checking  whether  the  training  system  is  effective  is  a  very  important  phase  in  training.  Due  to 
the  fact  that  measuring  the  results  of  team  training  is  more  complicated  than  it  is  for  training 
individual  operators,  this  activity  usually  is  not  performed  in  an  objective  and  accurate 
manner  (Salas,  Bowers  &  Cannon-Bowers,  1995;  Salas,  Morgan  &  Glickman,  1987;  Guerette 
et  al.,  1987).  Feedback  is  provided  at  the  level  of  the  individual  trainee,  rather  than  including 
the  team  aspects  of  the  performance.  Besides,  feedback  related  to  both  the  taskwork  and 
teamwork  (see  the  TEAM  model:  Morgan  et  al.,  1986;  Salas,  Morgan  &  Glickman,  1987) 
should  be  presented  to  the  team  members  (Guerette  et  al.,  1987).  Training  assessment 
procedures  for  collecting  team  performance  measurement  data  should  be  developed.  Given 
that  technological  advanced  learning  environments  are  developed,  the  possibility  of  automatic 
team  performance  measurement  (i.e.  both  collection  and  interpretation)  is  worthwhile  to  be 
explored  (Salas  &  Cannon-Bowers,  in  press;  Guerette  et  al.,  1987). 

The  cost-effectiveness  of  a  training  system  is  being  neglected  in  many  cases.  However,  in 
times  of  decreasing  budgets,  detailed  information  on  the  costs  of  training  and  instruction  must 
be  made  available  to  ensure  cost  effective  training.  This  is  especially  the  case  when  training 
devices  are  used  that  are  relatively  expensive  to  design  and  produce,  such  as  software 
packages  and  simulators  (Van  Berio,  1996b).  A  problem  in  estimating  the  cost  effectiveness 
of  a  training  system  is  the  difficulty  of  clearly  defining  the  precise  effects  or  benefits;  e.g.  it  is 
impossible  to  transform  these  in  monetary  value  only  (Blomberg,  1989).  Cost-effectiveness 
analysis  regarding  the  development  of  team  training  systems  is  a  largely  unknown  domain. 

The  guidelines  previously  discussed  differ  in  comprehensiveness.  Many  aspects  regarding  the 
development  of  team  training  are  being  included  (Salas,  Cannon-Bowers  &  Blickensderfer, 
1995),  but  a  coherent  set  of  guidelines  is  still  missing.  Also,  the  available  guidelines  are  not 
exhaustive.  Therefore,  a  consistent  methodology  for  developing  team  training  systems  has  to 
be  developed,  containing  prescriptive  guidelines  for  both  instructional  developers  and 
instructors. 
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Finally,  most  of  the  guidelines  and  methodologies  have  not  been  tested  empirically  (Salas, 
Bowers  &  Cannon-Bowers,  1995),  although  this  occurs  more  and  more  (e.g.  Brannick  et  al., 
1995;  Bowers,  Baker  &  Salas,  1994;  Fowlkes  et  al.,  1994).  Yet,  this  point  of  critique  applies 
equally  to  training  systems  directed  at  individual  trainees  (e.g.  Gustafson,  1991;  Andrews  & 
Goodson,  1980). 


6  FIELD  STUDY 

Besides  what  is  described  in  the  literature,  another  source  of  information  is  the  current 
practice  of  designing,  developing  and  executing  team  training.  In  this  chapter,  a  field  study  is 
briefly  described.  For  a  more  detailed  discussion,  see  Van  Berio  (1997b). 

The  primary  research  method  in  this  field  study  is  to  conduct  interviews  with  those  persons 
responsible  for  designing  and  executing  team  training.  Other  sources  of  information  (Yin, 
1984)  are  documents/archival  records  (e.g.  reports  of  interviews  and  visits  of  colleagues), 
observation^  (e.g.  attending  team  training  programmes),  and  physical  artifacts  (e.g.  training 
simulators).  For  practical  reasons,  only  Dutch  facilities  have  been  visited. 

Given  the  goal  of  the  field  study,  the  training  developers  and  the  trainers  are  appropriate 
persons  to  be  interviewed.  The  following  categories  of  interview  questions  have  been 
developed:  background  information,  organization  and  premises,  analysis,  design  and 
execution,  performance  measurement  and  feedback,  instructional  methods  and  training 
devices,  evaluation  and  maintenance,  and  concluding  remarks.  At  the  beginning  of  the 
interview  a  short  explanation  defining  the  goal  of  the  interview  and  the  purpose  of  the 
research  project  was  presented.  The  questions  were  not  always  asked  literally,  but  served  as  a 
structure  to  guide  the  interview.  A  detailed  overview  of  all  questions  is  included  in  the 
Appendix. 

In  order  to  get  a  broad  picture  of  the  way  team  training  is  being  designed  and  executed,  both 
military  and  civil  organizations  are  included  in  this  field  study.  The  primary  focus,  however, 
is  on  the  military  practices.  Interviews  were  conducted  with  persons  of  the  Royal  Netherlands 
Air  force,  the  Royal  Netherlands  Navy,  the  Royal  Netherlands  Army,  and  the  Royal  Military 
Police.  Due  to  the  fact  that  many  different  military  organizations  were  willing  to  participate, 
only  one  civil  organization  could  be  included  in  the  field  study. 

Twelve  interviews  have  been  conducted  from  November  1996  till  May  1997.  The  field  study 
shows  that  there  are  many  activities  in  the  field  of  team  training.  The  general  picture. 


^  It  is  planned  to  select  one  or  more  training  centres  in  order  to  attend  and  observe  team  training  programmes.  This 
step  is  an  essential  step,  because  the  actual  training  practise  could  be  somewhat  different  from  the  designer’s  and 
trainer’s  intentions  (expressed  in  the  interviews).  To  get  a  clear  picture  of  a  team  training  programme,  observations 
will  last  for  at  least  several  days  during  a  longer  period  of  time  (depending  on  the  total  length  of  the  training 
programme).  Ideally,  the  trained  team  should  also  be  observed  during  a  field  exercise  in  the  operational 
environment. 
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emerging  from  the  field  study,  shows  that  these  activities  are  not  always  very  systematically 
organized.  The  discussion  of  the  results  is  structured  around  the  categories  of  the  interview 
questions:  background  information,  organization  and  premises,  analysis,  design  and 
execution,  performance-measurement  and  feedback,  instructional  methods  and  training 
devices,  and  evaluation  and  maintenance. 

Background  information 

The  interviewed  persons  have  different  jobs/assignments  in  the  field  of  team  training,  varying 
from  commander  of  a  training  section,  to  instructional  developer,  trainer,  and  consultant.  A 
wide  variety  of  teams  are  being  trained  at  the  training  centres,  like  tank  platoons,  infantry 
units,  ground-to-air  defence  crews,  cockpit  crews,  fire  fighting  and  damage  control  units  of  a 
frigate,  military  police  units,  and  air-mobile  crews. 

Organization  and  premises 

No  standard  methodology  for  developing  team  training  systems  is  being  used.  In  most  cases 
an  existing  methodology  for  developing  training  aimed  at  the  individual  operator,  is  used. 
Although  the  main  phases  (analysis,  design,  development,  implementation,  evaluation)  are  the 
same,  the  interviews  show  that  the  specific  steps  comprising  the  phases,  appear  to  be  quite 
different.  Especially  conducting  an  analysis  of  team  tasks,  designing  training  scenarios  for 
teams,  and  assessing  the  performance  of  the  team,  are  steps  that  require  teamtraining-specific 
knowledge. 

During  the  process  of  developing  team  training  systems,  in  most  cases  hardly  any  information 
is  systematically  being  stored.  The  way  of  gathering  information,  making  decisions  about  the 
contents  and  structure  of  training  programmes,  and  executing  and  assessing  the  training 
programme,  is  therefore  largely  determined  by  an  individual  instructional  developer.  This  has 
the  risk  in  it  that  most  knowledge  and  skill  will  disappear  with  the  reallocation  of  this  officer. 
This  means  that  there  is  no  opportunity  to  build  a  knowledge  base  within  the  organization, 
promoting  the  re-use  of  information  and  expertise,  and  a  critical  reflection  on  the  process  of 
developing  team  training  systems.  Furthermore,  using  a  standard  methodology,  and  re-using 
information  enhances  the  efficiency  of  training  development. 

Analysis 

No  explicit  analyses  of  team  tasks  are  conducted.  The  tasks  of  individual  operators  are 
analysed,  sometimes  taking  into  account  the  specific  skills  for  operating  in  a  team.  Usually 
management  documents,  depicting  combat  instructions  of  (larger)  teams,  are  used  to  analyse 
the  team  and  the  operational  environment.  But  with  respect  to  developing  training,  these 
documents  are  too  general.  Therefore,  the  instructional  developer  has  to  rely  on  his  own 
operational  practice  in  the  particular  field.  A  disadvantage  of  merely  relying  on  this 
experience,  is  the  fact  that  every  expert  has  his  own  subjective  experience,  depending  on  the 
situations  in  which  he  performed  his  tasks.  This  experience  does  not  necessarily  has  to  be 
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representative  for  other  experts  as  well.  There  is  hardly  an  objective,  agreed-upon  base  for 
developing  team  training  programmes. 

Another  problem  of  merely  using  the  developer’s  operational  experience  rises  in  the  case  of 
new  kinds  of  teams.  These  newly  formed  teams  do  not  have  yet  performed  any  missions,  so 
there  are  no  experts  in  the  field.  Deciding  what  to  train,  and  which  performance  measures 
should  be  taken  into  account,  is  not  possible  on  the  basis  of  existing  operational  expertise. 

Design  and  execution 

There  are  no  guidelines  to  support  the  design  of  training  scenarios.  For  both  live  (field) 
exercises  and  simulator  training  sessions  the  instructional  developers  indicate  a  need  for 
support.  Designing  a  scenario  that  is  adequate  for  all  trainees,  taking  into  account  all  relevant 
practical  constraints  (like  personnel,  time,  available  terrain,  the  specific  training  device)  is  of 
paramount  importance  for  training  effectiveness. 

Besides  the  effectiveness,  the  efficiency  of  training  design  and  execution  can  be  improved.  An 
example  of  a  team  training  strategy  that  has  been  applied  is  that  all  team  members  simply 
receive  training  in  all  different  individual  tasks,  so  everyone  should  be  able  to  perform 
effectively  in  every  position  in  the  team.  In  another  case  the  student-commanders  play  the 
roles  of  the  other  team  members.  These  examples  look  like,  but  are  not  the  same  as, 
crosstraining.  Crosstraining  means  that  team  members  receive  training  in  their  own  tasks,  but 
also  receive  additional  training  in  other  team  members’  tasks.  In  the  previous  example, 
however,  no  formal  instruction  is  given  and  the  trainees  merely  act  as  other  team  members 
for  training  the  student  that  actually  plays  the  role  of  the  commander  at  that  time.  Also,  at  the 
respective  training  centres  all  team  members  received  training  relevant  to  all  tasks  of  all  team 
members.  But  the  degree  of  crosstraining  should  be  varied,  depending  on  the 
interdependencies  between  the  tasks  of  the  particular  team  members.  Applying  the  most 
adequate  crosstraining  strategy  could  take  less  time,  without  decreasing  results  (Schaafstal  & 
Bots,  1997). 

Performance  measurement  and  feedback 

In  order  to  conclude  whether  a  team  (or  a  team  member)  has  acquired  the  knowledge  and 
skills  required  for  operational  team  practice,  the  performance  should  be  assessed.  Therefore 
team  performance  measures  are  needed.  These  performance  measures,  however,  have  not  yet 
been  adequately  developed.  There  are  no  strict  criteria  for  assessing  the  team’s  performance 
in  a  more  objective  way.  The  consequence  is  that  every  trainer  has  considerable  degrees  of 
freedom  in  determining  whether  a  training  has  been  successfully  followed,  using  his  own 
operational  experience  as  point  of  reference.  This  has  the  risk  in  it  that  the  performance 
assessment  can  be  biased. 

Especially  when  training  larger  teams,  the  chief  instructor  is  not  able  to  monitor  and  assess 
the  team’s  performance  during  the  training.  Therefore,  assistant  instructors  (observers)  are 
being  deployed.  The  observers’  deployment,  however,  could  be  more  effective  when 
appropriate  support  is  provided.  Clear  instructions  concerning  their  tasks,  guidelines  for 
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monitoring  the  trainees’  performance,  guidelines  for  providing  feedback,  and  suitable  job-aids 
could  enhance  the  observers’  effectiveness. 

Instructional  methods  and  training  devices 

In  current  learning  environments  (real  life  and  simulated)  it  is  not  possible  to  train  teams  in  all 
relevant  skills.  On  the  one  hand  the  available  exercise  terrains  are  not  sufficiently  large  to 
realistically  conduct  life  exercises  deploying  larger  teams.  On  the  other  hand  the  present 
training  devices  can  not  simulate  all  important  operational  environments  and  events.  A  related 
point  is  the  fact  that  instructional  features  of  a  training  simulator  are  often  neglected  in  favour 
of  the  fidelity  features  (does  the  simulator  resemble  the  real  system  in  its  real  environment). 
Instructional  features,  like,  for  instance,  performance  measurement,  freezing  the  scenario, 
play-back  of  (parts  of)  the  scenario,  and  (automatically)  providing  feedback,  are  elements  that 
facilitate  learning  by  the  team  members.  Both  fidelity  and  instructional  features  should  be 
important  elements  of  a  (simulated)  learning  environment.  In  most  cases,  however, 
instructional  developers  are  not  involved  in  defining  the  specifications  of  training  simulators. 
Involving  instructional  developers  in  the  process  of  specifying  simulators  could  improve  the 
effectiveness  of  these  training  devices. 

Evaluation  and  maintenance 

The  effectiveness  of  training  programmes  is  hardly  measured.  Some  team  building  courses  do 
have  a  kind  of  interview  with  the  former  trainees  some  time  after  the  training.  In  other  cases 
the  training  centres  mail  a  questionnaire  containing  questions  about,  amongst  others,  the 
contents,  structure  and  relevance  of  the  training.  A  systematic  and  empirical  study  of  the 
effects  of  a  training  programme  is  not  conducted.  This,  however,  is  the  ultimate  check 
whether  the  training  programme  meets  its  goal  (i.e.  enhancing  the  team’s  effectiveness  in  the 
operational  environment). 

The  feedback  loop  from  operational  practice  to  training  programmes  is,  in  general,  not  a 
systematic  one.  It  remains  too  implicit  and  too  much  depends  on  coincidental  contacts 
between  persons.  Only  if  severe  flaws  are  found,  or  accidents  occur,  feedback  is  given  to  the 
training  centre. 


7  CONCLUSIONS 

The  results  of  the  literamre  and  field  study  show  no  contradictions.  Especially  on  the 
following  aspects,  both  studies  come  up  with  similar  results: 

•  A  general  and  encompassing  methodology  for  developing  team  training  systems  is  not 
available,  but  is  very  much  wanted  for.  There  are  many  activities  in  the  field  of  team 
training;  but  these  activities  are  not  very  structured,  and  the  officers  have  no  clear 
guidelines  that  can  be  of  help. 
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•  Conducting  a  task  analysis  is  essential  in  specifying  the  instructional  objectives.  The 
instructional  objectives  are  the  basis  of  the  actual  development  of  a  training  system.  With 
respect  to  team  training,  however,  no  proper  task  analyses  are  being  conducted,  resulting 
in  an  inadequate  specification  of  the  instructional  objectives. 

•  The  lack  of  explicit  instructional  objectives  hinders  the  development  of  team  training 
scenarios.  A  clear  description  of  the  to-be-trained  knowledge  and  skills  of  the  team 
members  facilitates  the  design  of  various  training  scenarios.  Guidelines  concerning  the 
application  of  several  team  training  strategies,  like  crosstraining,  could  enhance  the 
efficiency  of  the  training  programme. 

•  Measuring  the  performance  of  a  team  and  providing  feedback  are  crucial  in  team  training. 
But  these  aspects  of  team  training  development  are,  at  the  same  time,  the  most  complicated 
ones.  In  only  one  case  (the  training  of  a  two-person  cockpit  crew)  a  rather  sophisticated 
measurement  and  feedback  system  has  been  developed.  In  all  other  cases,  the  issue  of 
performance-measurement  and  feedback  raised  many  questions,  and  no  explicit  answers. 

•  The  specification  of  training  simulators  is  more  than  merely  copying  the  actual  system  or 
actual  task  environment  into  a  learning  environment.  Rather  than  emphasizing  the  fidelity 
of  these  learning  environments,  the  instructional  features  should  be  taken  into  account. 
Involving  instructional  developers  and  trainers  in  this  process  could  improve  the 
effectiveness  of  these  training  devices. 

•  Finally,  assessing  the  quality  of  the  training  programme  is  hardly  being  done.  This  is  an 
essential  step,  however,  in  determining  whether  the  actual  training  need  has  been  really 
met.  Rather  than  merely  waiting  for  feedback  from  the  operational  officers,  instructional 
personnel  could  anticipate  new  developments  and  changing  instructional  needs.  Guidelines 
could  support  also  in  this  respect. 

It  is  planned  to  select  one  or  more  training  centres  in  order  to  attend  and  observe  actual  team 
training  programmes.  This  step  is  an  essential  step,  because  the  actual  training  practise  could 
be  somewhat  different  from  the  designer’s  and  trainer’s  actual  intentions  (expressed  in  the 
interviews).  To  get  a  clear  picture  of  a  team  training  programme,  observations  will  last  for  at 
least  several  days  during  a  longer  period  of  time  (depending  on  the  total  length  of  the  training 
programme).  Ideally,  the  trained  team  should  also  be  observed  during  a  field  exercise  in  the 
real  operational  environment. 

The  results  of  the  literature  and  field  study  will  be  analysed  and  integrated  into  a  (paper- 
based)  prototype  of  a  methodology  for  developing  team  training  systems.  This  prototype 
methodology  has  to  contain  guidelines  as  concrete  as  possible.  Further  research  will  be  aimed 
at  developing  the  distinct  guidelines  comprising  the  methodology,  and  their  empirical 
validation. 
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APPENDIX  Interview  questions 


Introduction 

•  To  what  extent  does  designing  instruction  for  individual  operators  differ  from  designing 
instruction  for  teams? 

•  Is  there  a  need  for  a  methodology  specifically  aimed  at  developing  team  training? 

•  The  literature  is  not  very  specific  on  this  topic. 

•  The  purpose  of  this  field  study  is  to  gain  insight  into  the  way  team  training  is  being 
designed  within  the  military.  That  is  why  I  have  chosen  your  organization. 

•  It  could  be  that  some  questions  are  being  asked  that  seem  rather  strange  for  you,  and  that 
can  not  be  answered.  That  is  alright.  I  am  just  interested  in  your  organization’s  specific 
way  of  handling  team  training. 

Background  information 

1  Could  you  tell  something  about  your  own  professional  background? 

2  How  many  training  developers  and  instructors/trainers  are  working  in  this 

organization? 

3  How  many  teams  (trainees)  a  year  are  being  trained? 

4  What  kinds  of  training  are  being  developed/executed? 

Organization  and  premises 

5  How  is  the  organization  of  developing  team  training  organized  (one  person, 

development  team,  other  persons)?  How  is  the  coordination  between  these  persons 
organized? 

6  Is  a  team  training  program  being  executed  by  only  one  instructor/trainer,  or  more?  If 
the  latter  case  is  true,  how  is  the  coordination  between  the  instructors  being  organized? 

7  What  (general)  principles  for  instructional  systems  development  are  being  applied? 

8  Is  the  development  process  being  supported  by  some  methodology  handbook, 
prescriptions,  and/or  software  tools? 

Analysis 

9  How  is  the  training  need  established? 

10  How  are  team  tasks  being  analysed? 

1 1  How  are  instructional  objectives  being  derived? 

Design  and  execution 

12  How  is  the  structure  of  the  training  program  (learning  trajectory)  established?  This 
relates  to  the  structure  of  the  team  training  on  the  one  hand,  and  the  relationship 
between  team  training,  individualized  training  and  the  operational  practice  on  the  other 
hand. 

13  How  are  training  scenarios  being  developed? 

14  Are  there  certain  team  characteristics  which  influence  how  to  instruct  a  team  (e.g.  the 
way  of  communication,  kind  of  leadership,  mutual  personal  relations)?  If  so,  which  are 
these,  how  are  these  identified,  and  how  does  this  adapt  the  instruction? 

15  To  what  extend  is  instruction  being  differentiated  towards  the  individual  team  members, 
and  how  is  this  accomplished? 

16  How  is  checked  for  the  involvement  of  all  team  members  during  the  execution  of  the 
training  program? 

17  How  do  you  monitor  the  learning  process  of  the  team  and  the  individual  team 
members? 

18  How  do  you  decide  when  to  intervene  in  the  training  process? 
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Performance  measurement  and  feedback 

19  Is  the  performance  of  the  team  assessed  during  the  training?  If  so,  at  what  moments? 

20  How  is  the  team  performance  being  measured?  To  what  extend  is  there  a  distinction 
between  the  process  and  product  of  the  team  performance? 

21  Is  the  individual  team  member’s  performance  being  measured?  To  what  extend  is  there 
a  distinction  between  the  process  and  product  of  the  individual  team  member’s  perform¬ 
ance? 

22  Is  feedback  being  provided  towards  the  team  as  a  whole,  towards  the  individual  team 
members,  or  both  to  the  team  and  its  individual  members? 

23  How  is  the  feedback  being  provided? 

24  Is  a  debrief  at  the  end  of  a  team  training  being  organized?  If  so,  how  is  this  done? 

Instructional  activities  and  training  devices 

25  Which  instructional  activities  are  being  employed  during  the  team  training? 

26  Which  training  devices  are  being  applied  during  the  team  training? 

27  How  are  the  requirements  for  (technologically  advanced)  training  devices  being 
specified?  Who’s  responsibility  is  this? 

Evaluation  and  maintenance 

28  Is  the  effectiveness  of  the  training  program  established?  If  so,  how  is  this  being  done? 

29  Is  there  a  feedback  loop  from  the  operational  practice  back  to  the  training  program?  If 
so,  how  does  this  work? 

30  Are  sufficient  resources  available  in  order  to  maintain  training  programs,  scenarios  and 
training  devices? 

31  In  what  way  are  changing  instructional  needs  being  encountered? 

Concluding  remarks 

32  Are  there  specific  aspects  (bottlenecks,  positive  points)  regarding  the  development  of 
team  training  systems  that  have  not  been  discussed  yet? 

33  Do  you  have  any  remarks  and/or  questions  with  respect  to  this  interview? 
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De  onderzoeksvraag  van  deze  studie  is  uit  welke  richtlijnen  een  opleidingsontwikkelsystematiek 
voor  team  trainingen  zou  moeten  bestaan.  Om  deze  vraag  te  kunnen  beantwoorden  wordt  de 
volgende  strategie  voorgesteld;  uitvoeren  van  een  literatuurstudie,  uitvoeren  van  een  veldstudie, 
ontwerpen  van  een  prototype,  uitvoeren  van  een  expert-evaluatie,  uitvoeren  van  diverse  experi- 
menten  om  het  prototype  te  testen,  en  toepassen  van  de  systematiek  in  een  of  meerdere 
praktijksituaties.  Deze  strategie  wordt  uitgebreid  toegelicht.  De  resultaten  van  de  literatuurstudie 
(Van  Berio,  1996a)  en  de  veldstudie  (Van  Berio,  1997b)  worden  kort  besproken. 
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